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I. INTRODUCTION. 



The simple fractional flow (or Markov-type) model of personnel movements 
through an organization has been widely analyzed (see for example Bartholomew 
(1973), Blumen, Kogan and McCarthy (1955), Lane and Andrew (1955), Rowland and 
Sovereign (1969) and has been widely applied, especially in military manpower 
planning (see U.S. Navy (1973)). Other models such as the "cohort" and "chain" 
models (see Marshall (1973)) and Grinold and Marshall (to be published)) satisfy 
more realistic assumptions on personnel movement, but lack the convenient struc- 
ture of the Markov model. The purpose of this paper is to present an extension 
of the Markov Model to one with a 2-dimensional state space. The state space 
is chosen so that the fractional flow matrix has a special structure which is 
then exploited. 

In section II the structure of the model is presented and in section 
III examples are given. In section IV we present the probabilistic properties 
of the model with emphasis on computationally tractable formulae. In section 
V the structure of the model is exploited in the personnel stock and flow 
equations . 

II. STRUCTURE OF THE FRACTIONAL FLOW MATRIX. 

We assume that for planning purposes an organization considers time 
in discrete periods, and that people are counted at the end of each period. 

When counted, a person is assumed to possess two characteristics i and j, 
and is said to be in state (i,j), where i represents the first character- 
istic (FC), 1 ^ i £ n, and j represents the second characteristic (SC), 

£(i) £ j ^ u(i). Here £(i) and u(i) are the lower and upper limits 
respectively for the SC when i is the FC. Also let J(i) = { j | £ (i ) ^ j ^ u(i)}, 
the set of SC f s for FC i, and let w. be the number of elements in J(i). 

l 

Let q^(j,m), j,m £ J(i) be the fraction of people in state ( i , j ) 
in a time period who move to state (i,m) in the next time period, and let 



Q be the x matrix [ ( j , m) ] . Let P i (j,m), j € J(i), m 6 J(i+1) 

be the fraction of people in state (i,j) in a time period who move to state 
(i+l,m) in the next time period, and let be the w_^ x w^ + ^ matrix 

[p^(j,m)]. A basic assumption of our two-characteristic model is that move- 
ment in one time period from states with FC i can only be to states with FC 
i or i+1, or out of the system. Following the notation of earlier papers, 
let Q be the fractional flow matrix for all active states in the system. 
Then Q has the following structure: 



Q = 



(l) 



Vl P n-1 



where the zero matrices have been suppressed. Throughout this paper we assume 

that people can leave the system eventually from any state and thus ( I— Q) 

has an inverse, where I is the identity matrix. This inverse (I-Q) ^ is 

called N, the fundamental matrix (see Kemeny and Snell (I960)). For each i 

let = (I-Q^-P^)l, where I is a vector with every element equal to 1. 

Then A. is a vector of w. elements, each one an attrition fraction from 
l i 

the appropriate state. 



III. EXAMPLES. 

a) The LOS Model . 

Let the "length of service" that a person has completed with an organ- 
ization be denoted LOS, and let the FC of a person denote his LOS. If the LOS 
is measured in the same time units as the planning periods then each Q_^ matrix 
is a zero matrix. In each planning period a person’s LOS must increase by one 
unit, so that Q has the structure 



2 



0 P 



1 

0 P, 



Q = 



\ 

\ 

\ P 

\ 



n-1 



\ 



By appropriate choice of the second characteristic P^ often has special 
structure too. Consider a hierarchical system where the "rank" or ’'grade" of 
an individual is used as his second characteristic. Then the structure of 
each P^ depends on the organization’s promotion scheme. Assume that in one 
period a person either stays in the same grade or is promoted one grade. No 
demotions occur. Then P. would have the structure 

l 



P. = 

l 




X X 
\ \ 

\ ' 



\ 



\ 



\ 



where x represents a non-zero element, 
b) The (Grade, LOS) Model . 

If, as in a) no demotions occur, and only single promotions can occur 
in a time period, then if i indexes the grade, and j the length of service, 
then Q has the structure shown in (1) . Each submatrix has special structure 
also. Let 
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q = probability a person in state (i,j) at end of one period 
will be in state (i,j+l) at the end of the next period, 
p_ = probability a person in state (i,j) at the end of one 

period will be in state (i+l,j+l) at the end of the next 
period. 



The transition matrix has non-zero elements only immediately 

above the main diagonal: 






0 



0 



q i,£(i)+l 

0 



q i,u(i)-l 

0 



The transition matrix P^ has non-zero elements only on a single 
diagonal band. If £ (i+1) ^ £(i) + 1 and u(i+l) ^ u(i) + 1, then P^ 
the form shown below, where: 

1) the top max {0 ,£ (i+l)-(£ (i)+l) } rows are zeros, 

2) the last max {0 ,u(i+l)- (u (i)+l) ) columns are zeros. 



P. 

i 



0 

0 

p i,£(i+D-i 

0 P i,£ (i+1) 

0 



P i,u(i) 



has 
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If £(i+l) £ £ (i) , the first &(i)+l-& (i+1) columns of P are zeros. If 
u(i+l) ^ u(i), the bottom u(i)-Pl-u(i+l) rows of are zeros. Under any 

circumstances P^ has only one non-zero diagonal, and we call such a matrix 
a diagonal matrix. Efficient methods of storing and multiplying diagonal 
matrices are discussed in Hayne and Marshall (1974). 



c) The (Grade, TIG) Model . 

In certain applications a person’s "time in grade, n denoted TIG, is 
more important than his time in the system (LOS). If again we allow no demo- 
tions and only single promotions per period and if the FC indexes the grade 
and the SC the TIG for the appropriate grade, then Q has the same structure 
(1) . Each has the same structure as in b), but now £(i) = 1 for each 

grade i. However, each matrix P^ has a single column of non-zeros. 



P. 

i 




0 



P. 

i 



>u(i) 



0 



0 



0 



since promotions to the next highest grade lead always to a TIG of 1. Here 
p__ is the fraction those in grade i, with time in grade i equal to j, 
who are promoted to grade i+1. 

If demotions are not allowed and only single promotions can occur per 
period, then grade is a characteristic which can be used as the FC. A larger 
number of possibilities occur for the SC. In addition to those above some 
useful ones are (i) skill category, (ii) physical location in a multi-location 
organization, and (iii) educational level. Note that educational level could 
also be used as the FC. 
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IV. SOME PROBABILISTIC PROPERTIES. 



Let T^ be the set of states associate with FC i; thus 

T i = ( (i > j ) | j ^ J(i) } , i = 1,2, ...,n. 

Also let = u(i) - £(i) + 1, the number of states in T\ (and J(i)). 

Finally let Tq be the single state ,f out of the system." 

In this section we develop the probabilistic properties of: 

1) any set of states T\ , 

2) any union of consequtively indexed sets 

m 

i .e . U T., 
i=k 1 

3) the union of all transient states, which we call T. 

One of the purposes of this development is to show that the stochastic proper- 
ties of Q, typically a large matrix, are readily calculated in terms of the 

smaller matrices Q. and P., and as seen in section III these often have 

i i 

extremely simple structure which leads to simple computation. 

The format of this section follows closely that of Chapter 3 of Kemeny 
and Snell (1960). The notation (K&S,3._._) indicates that a result follows 
from theorem 3._._ in Kemeny and Snell, albeit usually not directly, 
a) First-Order Properties . 

Recall that we assume system matrix Q has a fundamental matrix 
N = (I-Q) \ and each element of N is the expected number of visits to 
the column state starting from the row state (K&S,3.2.4). Since Q has the 
structure shown in (1) , 
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N NPN NPNPN 
1 112 112 2 3 



N NPN 

2 2 2 3 



N = 



N„ 



n-1 

n (n.p.)n 

i=l 1 1 n 

n=l 

n (n.p.)n 

i=2 1 1 n 

n-1 

. n , (N.P.)N 
1=3 l l n 



( 2 ) 



n 



where = (I-Q^) ^ , i = l,...,n, the fundamental matrix for FC i. Note 

that the large matrix N is completely determined by the matrices and 

P^. Thus the only matrix inversions required are those of (I-Q^) , i = l,...,n. 
This is of considerable computational significance because as previously men- 
tioned Q is usually a large matrix. 

Each matrix has a probabilistic interpretation. We pursue this 

interpretation and show that these matrices can be used to determine other 
probabilistic properties of interest. 

In this section we make numerous definitions and denote the k^- one 



by Dk. 

Let us consider first the properties associated with a single set of 
states T. and define: 

i 

Dl. v^(j,m) = expected number of visits to state (i,m) given that FC i 
is entered in state (i,j), 

D2. = a w^ x matrix having v^(j,m) as the element in row 

j-£(i)+l and column m-£(i)+l. 



From (2) the element of in row j-£(i)+l and column m-£ (i)+l 

equals the expected number of visits to state (i,m) given that FC i is 
entered in state (i,j). (K&S,3.2.4). So, from definitions Dl and D2 , we have, 
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( 3 ) 



Note that the rows and columns of N. and V. correspond to states in 

11 

lh in the same order as the rows and columns of Q^. 

Now define: 

D3. x^(j) = expected time in FC i given that FC i is entered 

in state (i,j) , 

D4. t ± = [T i (i(i)),...,T 1 (u(i))] > a w ± x 1 vector. 

The expected time spent in FC . i equals the sum of the expected number of 
visits to the various states in FC i. From (3) and D3, 

T^(j) = component (j-£(i)+l) of N^l, 

and from D4 , 

t. = N.l, a w. x 1 vector, (4) 

i i l 

where 1 is a vector with all components equal to one. 

We next turn our attention to where the process goes when it leaves 
FC i. The process upon leaving must enter either (if i < n) or 

T^. Next define 

D5. b^Cjjin) = probability of entering FC i+1 in state (i+l,m) given 
that FC i is entered in state (i,j), 

D6 . B ± - a w^ x w^_^ matrix having b^(j,m) as the element in 

row j-£(i)+l and column m-£(i+l)+l, 

D7. b^(j) = probability of ever entering given that FC i is 

entered in state (i,j), 

D8. b^ = [b^(£ (i) ) , . . . ,b^(u (i)) ] , a w i x 1 vector. 
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given that FC i 



D9. b^(j) = of never entering T^_^ 

is entered in state (i,j), 

DIO. b iQ = [b ±0 ( Jl (i) ) , . . . ,b i() (u(i) ) ] , a w ± * 1 vector. 

From these definitions it follows that 



B. = N.P., a w. x w. matrix (K&S,3.5.4), (5) 

111 11 

b. = B.l, a w. x 1 vector, 
ii l 

and 




= N.A., a w. x 1 vector. 

ii l 

The matrix B_^ is particularly useful in manpower policy analyses. 

For example let f ^ be a 1 x vector of the number of people entering 

T. . Then f.B. is a 1 x w. M vector of the number of these people who will 
l ii l+l 

eventually enter T^ + ^. (K&S,3.3.6). Such applications will be the subject 

of future papers. 

Next we consider the first-order properties related to FC*s i and 
k where i £ k. Define: 

Dll. b ( (i, j ) , (k,m) ) = probability of entering FC k in state (k,m) 

given that FC i is entered in state (i,j), 

D12. B^ = a w^ x matrix having b ( (i , j ) , (k,m) ) as the 

element in row j-£(i)+l and column m-£(k)+l. 



From definitions D5 and Dll and a simple conditioning argument we have, 

u(i) 

b ( (i, j ) , (i+2 ,m) ) = J b ( j , r)b (r ,m) . 

r=£ (i) 1 1+1 

So, from D12, 



B . . „ - B . B . 0 . 

l , i+2 l 1+2 
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Notice from Dll that B. . is an identity matrix and from D5 that B. .... = B.. 

11 i,i+l 1 

More generally it can be shown that for i £ k, 

k-1 

B.. = n B , a w. x w. matrix, 
lk r l k 

r=i 

Define: 

D13. v( (i, j ) , (k,m)) = expected number of visits to state (k,m) given 

that FC i is entered in state (i,j), 

D14. V = a w. x w matrix having v( (i , j ) , (k ,m) ) as the 

IK. IK 

element in row j-£(i)+l and column m-£(k)+l, 

D15. b^O) = Probability of ever entering FC k, given that 

FC i is entered in state (i,j), 

D16. b^ = [bik(^ (i) ) , • • • >b^^(u(i) ) ] , a w^ x l vector. 

Considering each row of B^ as the part of an initial probability vector 
that applies to T^, we then have, 

V = B.,N , a w. x w matrix (K&S,3.5.4), (6) 

IK IK K 1 K 

and, 

b #1 = B.-I, a w. x 1 vector, 

lk lk i 

Define: 

D17 . = expected time in FC k given that FC i was entered 

in state (i,j) , 

D18 ‘ T ik = ^ik^ 1 ^ ’ ' * * > T ik ( u ( i )) 1 » a w k x 1 vector * 

The expected time in an FC is the sum of the expected number of visits to 
states in that FC, so 

t . , = V., 1, a w. x 1 vector, 

lk lk l 
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This completes our study of the first-order properties related to the 
various FC*s of the system. The foregoing definitions by no means exhaust 
the first-order properties of the two-characteristic model that might con- 
ceivably be of interest. It is felt, however, that these properties will 
often be of practical interest and that other first-order properties may be 
readily derived from those given above. 

b) Two Special Cases . 

The elements of the fundamental matrix for FC i, have a somewhat 

different interpretation when the states in FC i have what we call the "0-1 
visiting property. 1 * We say that a state has the 0-1 visiting property if 
the state can be visited no more than one time. Important examples of two- 
characteristic models in which all transient states have the 0-1 visiting 
property are the models in which the FC or SC is length of service or where 
the SC is time in grade. 

If each state in T^ has the 0-1 visiting property, then the expected 
number of visits to a state in T^ is equal to the probability of visiting 
the state. The element of in row j-&(i)+l and column m-£(i)+l may 

then be interpreted as the probability of visiting state (i,m) given that 
FC i is entered in state (i,j). 

Another property of interest is the "no return property.** We say that 
a set of states has the no return property if it is impossible to ever make a 
transition into the state after a transition has been made out of the state. 

The 0-1 visiting property implies the no return property, but they are not 
equivalent. For example, in modelling manpower flows in the U.S. Civil Service 
one might use *'GS grade** as the FC and "pay step" as the SC. Each state is 
then a couple (grade, pay step). A person can stay in the same pay step for 



11 



more than one period, so if there are no demotions then each state would have 
the no return property but not the 0-1 visiting property. 

If the states in T\ have the no return property then it is possible 
to order the states in 1\ so that is upper triangular. When is 

upper triangular so is I-Q^ and the computation of the inverse of I-Q^, 
i.e. the fundamental matrix for FC i, ISL , is considerably easier than in 
the general case. 

If the states in 1\ have the 0-1 visiting property, then not only 
is upper triangular but also the elements of on the main diagonal 

are all ones. 

c) Variances . 

The format in this section follows closely that of section a), but here 
we are concerned with various second moment properties of the two-characteristic 
model. 

Define: 

D19. v .(j,m) = variance of the number of visits to state (i,m) given 
z , 1 

that FC i is entered in state (i,j), 

D20. V. . = a w. x w. matrix having v^ . (i,m) as the element 

2,i ii 2,i 

in row j-£(i)+l and column m-£(i)+l. 

Following (K&S ,3.3.3), 



V . = N. (2 (N. ) -I) - (N.) , 

2,1 i v l dg i sq 



where for any square matrix A, A. and A both have the same dimensions 

dg sq 

as A; A^ is defined when A is square and is formed by setting all elements 
in A not on the main diagonal to zero; A g ^ is formed by squaring all the 
elements in A. 
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Define : 



D21. . (j) = variance of the time spent in FC i given that FC i 

z , 1 

is entered in state (i,j) 

D22. t 2 ± = [t 2 i (£(i)) , • • • ,t 2 ± (u(i))], a w i x 1 vector. 



Following (K&S,3.3.5), 



2 ,i 



(2N.-I)t . 
11 



(x.) . 

l sq 



Define : 

D23. v^ ( ( i , j ) , (k,m) ) = variance of the number of visits to state (k,m) 

given that FC i is entered in state (i,j), 

D24. V^Cijk) = a w^ x w^ matrix having v^ ( (i , j ) , (k ,m) ) as 

the element in row j-£(i)+l and column m-£(k)+l, 



Following (K&S, 3.3.6), 



V.(i,k) = V M (2(N. ) , -I) - (V M ) . 

2 lk k dg lk sq 



Define : 



D26. T 2 ((i>j)>k) = variance of time spent in FC k given that FC i 

is entered in state (i,j), 

D27. x 2 (i,k) = [T 2 ((i,£(i)),k),...,T 2 ((i,u(i)),k)], a w ± x 1 

vector. 



Following (K&S, 3. 3.6) , 



T 2 ( i,k) - B. k (2S k -I)T k - (T. k ) aq . 



If each state in T\ has the 0-1 visiting property, then the diagonal 
elements of are equal to one, and, 
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<Vdg - *• 

V 2,i =N i - <V S q> 



V 2 (i ’ k) * V ilc - <Vsq' 



d) Matrices of t-Step Transition Probabilities . 

In this section we consider the probability of being in state (k,m) 
t steps after being in state (i,j). The matrices of these probabilities 
are called the t-step transition matrices. They are used in section V to 
represent the stock vectors as a sum of steady-state and transient components. 
Define: 

D28. m(t : (i , j ) , (k,m) ) = probability of being in state (k,ra) t steps 

after being in state (i,j), t = 0,1,2,... 

D29. M^(t) = a x matrix having m(t ; (i , j ) , (k ,m) ) as the 

element in row j-£(i)+l and column m-£(k)+l. 

The rows of M #1 (t) are associated with states in T. ; the columns of M #1 (t) 
lk 1 lk 

are associated with states in T . 

k 

We have immediately that 



M. . (0) = I. 

li 



From our assumptions on the structure 


of 


Q 


we have 


M lk (t) - 0 


if 


i 


> k. 


M. k (t) - 0 


if 


t 


< k-i. 



If the process is to be in state (k,m) exactly t steps after being 
in state (i,j), then it must be in some state with FC k or k-1 exactly 
t-1 steps after being in state (i,j). Conditioning on this fact leads to the 
recursive equation. 
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(7) 



M ik (t) = + M i,k-l (t-1)P i-l’ 



t = 1,2,... 



For any i and k the sum over t of the probability matrices M (t) 

1 KL 

gives the matrix of the expected number of visits to states with FC k start- 
ing from states with FC i. So we have, 



J 0 M ik (t) * v ik- 1 £ k - 



( 8 ) 



= 0 , 



otherwise . 



Recall that the Q matrices are transient, so V . is matrix of 

1 lk 

finite elements. This implies that, 



lim M (t) = 0. 

t-*» 1 

From (7) it can be shown by an inductive argument that 

H ik (t) ■ X Yk-i^-^VX- 

r=0 



(9) 



( 10 ) 



The t-step transition matrices provide a rather comprehensive picture 
of how people move through a two-characteristic system. 



e) Conditioning on Promotion When the FC is Grade . 

In manpower planning one is often interested in conditional probabilities, 
e.g. the probability of attaining grade k given that grade i is attained. 

The stochastic properties of the transient matrix Q under conditioning on 
promotion when the FC is grade are briefly developed in this section. 

Define : 

D30. (i,j;t) = the event "in state (i,j) at time t" 

D31. = the event "a transition is made into before leaving 

the system." 
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is the same as conditioning on promotion to 



* 

Conditioning on the event 
grade k. 

Define : 

D32 . (j ,m) * Pr[(i,m;t+1) | (i,j ;t)] 

D33. q^j.m) = Pr[(i,m;t+1) | (i,j ;t) ,T i+1 ] 

Provided that Pr | (i , j ; t) ] ^ 0, we have by conditional arguments, 

* b i (m) 

q i (j>m) = q i (j>m) x — ~ . (ii) 

Define : 

D34. C_^ = a w_^ x w_^ matrix having the elements of (see D8) on its 

main diagonal and zeros elsewhere. 



We assume that promotion to grade i+1 is possible from every state 
in 1\. Under this assumption exists. If promotion to grade i+1 is 

impossible from some state (i,j) then we must avoid conditioning on an 
impossible event. This is readily accomplished by temporarily treating state 
(i,j) as part of (out of the system) and redefining J(i), Q^, P^ and 

A_^ accordingly. 

Define : 

* * 

D35. Q_^ = a w^ x w_^ matrix having q^(j>ni) as the element in row 

j-£(i)+l and column m-£(i)+l. 



Then from (11) and D34, 



* -1 

Q. = C. Q.C. . 

l ill 



The matrix Q_^ is the matrix of within grade one-step transition probabili- 
ties conditioned on the attainment of grade i+1. 
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Define : 



D36 . 


P ± ( j >m) 


= Pr [ (i+1 ,m; t+1) 


1 (i , j ; t) ] 


D37. 


* 

P i (j > m > 


= Pr [ (i+1 ,m; t+1) 


| (i y j > t) , T^_^] 


D38 . 


* 

P i 


= a w. x w. , , 
l l+l 


* / X 

matrix having p^(j,ni) as the element 



in row j-£(i)+l and column m-&(i+l)+l. 



We then have, 



* 1 
P. (j >m) = p (j ,m) x 



t^Cj) 



Thus from D34 and D38 , 

* -1 

P . = C . P . . 

1 ii 

* 

The matrix is the matrix of one-step promotion probabilities conditioned 

on the attainment of grade i+1. 

Because (Q*) r = cTVc. , 

the fundamental matrix for grade i when we condition on promotion to grade 
i+1 is 

* * —1 

N ± = (I-Q.) 



* r 



- I (QV> 

r=0 



= l C.Vc. 

r=0 1 11 



= C. 1 N.C. . 
1 11 



Define : 

* 

D39. v^(j,m) = expected number of visits to state (i,m) given that 

grade i is entered in state (i , j ) and grade i+1 is 
attained . 
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D40. 



* 

V. 

l 

D41 . b i (j,m) 



D42. B* 
1 



= a w_^ x matrix having v^(j,m) as the element in 
row j-£(i)+l and column m-£(i)+l 
= probability of entering grade (i+1) in state (i+l,m) 
given that grade i is entered in state (i,j) and 
grade i+1 is attained. 

* 

= a w_^ x matrix having b^(j,m) as the element 

in row j-£(i)+l and column m-£(i+l)-l. 



Then one may show that, 



* * 




and 

* * * 

B. = N.P. 

l li 

= c7 1 b. . 

1 1 

* k 

Note that B_^ is simply B_^ with its rows normalized, but Q_^ is not 
simply a row normalized form of Q.. 

: k 

As with the matrices B_^, products of matrices B_^ with successive 
indices are well defined: their meaning is that of a matrix B ^ as defined 

in Dll and D12 with conditioning on attainment of grade k. 

The conditioned and unconditioned matrices may be used together. For 

k 

example, the elements of gi- ve the probabilities of entering grade 

i+2 in the column state conditioned on starting from the row state in T_^ 
and eventually attaining grade i+2. 



V. EQUATIONS OF STOCKS AND FLOWS. 

We begin by defining the terms "stocks" and "flows," and then discuss 
why stocks and flows are important in manpower planning models. Next the 
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relations between stocks and flows in a two-characteristic model are developed. 
Finally, we show how the stocks can be represented as the sum of a "steady- 
state M component and a "transient" component, 

a) Definitions and Background . 

A period is the interval of time from immediately after an integer 
value of the time parameter t up to and including the next integer value of 
t, A period is identified by the value of the time parameter at the end of 
the period. Thus, 

period t^ = { t : t^-1 < t ^ t^} 

where t^ is an integer. 

The number of people in a state at the end of a period is referred to 
as the "stock" in that state. Thus, stocks are counted only at integer values 
of the time parameter t. 

The number of people who change their status in the system from one 
state to another during any period is referred to as a "flow." Flows occur 
during a period, but we do not specify the exact time at which they occur. 

Stocks and flows are of primary importance in most manpower planning 
models. The most obvious reason for this is that costs are closely related 
to stocks and flows, e.g. total payroll depends on stocks, transportation 
costs or retraining costs depend on flows. Recruiting policy and promotion 
policy depend in the short term on present stocks and in the long term on how 
we model future stocks and flows. Determining the feasibility of a retirement 
plan and evaluating the effects of a change in billet structure are other 
instances in which the planner needs to be able to model stocks and flows in 
a manpower system. 
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We now define the variables that are used to model the stocks and 



flows in the two-characteristic model. Recall that T. is the set of states 

1 

associated with FC i, w. is the number of states in T. , and for conven- 

1 1 

ience of notation we assume the second characteristic takes on successive 
integer values for FC i. 

In a Markov model the stocks and flows are in general random variables 
In this section we deal only with the expected values of stocks and flows. 
Such a model is called a "fractional flow model" because the transition proba 
bilities of the Markov model are in effect treated as fractions which direct 
flows through the system in a deterministic manner. 

Let, 



s u (t) 



= expected stocks in state (i,j) at time t. 



s i <0 ’ s i,»(i) (t))l 

a 1 x vector of expected stocks in T\ , 



s ( t ) = (s (t),...,s (t)), 

i n 

n 

a lx Y w. vector of expected stocks in the 
i=i 1 

system. 



By our basic assumption, flows into any state in must come from a 

state in either T\ or T^_^. We also make provision in our model for 
"external flows." The source of such flows is unspecified. However, we may 
consider external flows as consisting of people hired into the system. The 
external flows may be deterministic or random, but we deal only with their 
expected values. 

Let , 
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d_(t) = expected flow from states in to state (i,j) 

during period t, a scalar; 



d i a> ■ (d i,«(i) (t) ’---> d i, u <i) a),> a 1 * "i vect ° r; 






expected external flow into state (i,j) during 
period t, a scalar; 



f i (t) = (f i,«(i) (t) f i,u(i) (t)) ’ a 1 x vector; 



gij(t) = ex P ectec * flow from states in to state (i,j) 

during period t, a scalar; 



g i (t) = (g M(i) (t) ’**-’ g i,u(i) (t)) ’ a lxw i vector< 

When i = 1, g_(t) I s defined to be zero. 

The relation between the flow vectors and the stock vector in grade i 
is depicted in Figure 1 where n T ;t" denotes the states with FC i at time 
t • 




Figure 1 , Stocks and Flow with FC i in Period t. 
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b) Basic Stock Equation . 



Clearly, from our assumptions, 

s i (t) = d i (t) + f i (t) + g.(t). 

(See Figure 1.) 

It will be convenient to define, 

s^(t) =0, a vector of zeros, 
Pq =0, a matrix of zeros. 

Using conditional expectation we then have 



d ± (t) = s i (t-l)Q 1 , 


r 1 , . . . , n , 


g i (t) ' s i-i (t " 1 )p i-r 


l l,...,n« 



The basic stock equation is then, 

s i (t) = s i (t-l)Q i + f ± (t) + s._ 1 (t-l)P._ 1 , i = (12) 

The basic stock equation for FC i can be written in terms of the 
expected or actual stocks with FC i in previous periods. By recursively 
applying the basic stock equation for s^ (t) ,s^ (t-1) , . . . , s^ (1) one obtains 

t-1 t-1 

s.(t) = S.(0)Q^ + l f.(t-r)Q^ + l s w (t-r-l)P q[, 

r=0 r=0 

t = 0,1,2,..., (13) 

i = 1, . . . ,n, 

which we will refer to as the cumulative stock equation . 

Equations (12) and (13) are used frequently in the remainder of this 
paper. Some manpower models used in the U.S. military for short-range fore- 
casting consist principally of an application of an equation similar to (12). 
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c ) Transient Properties of the Stocks . 



In this section we develop a method for expressing the stock vector 
as a sum of a "steady-state" component and a "transient" component. This 
method helps one to understand how the stock vectors change in going from 
any present stock vector to future stock vectors. This method also helps 
one interpret the character of the limiting stock vector. 

We do not want to restrict ourselves to cases in which the stock vector 
converges (as t increases) to a finite vector. We say that the vector 
function s^(t) is a steady-state component of the stock vector s^(t) if, 

lim (s . ( t ) — s . (t)) = 0. 

1 i 

For any sequence of stock vectors <s^(t)> there is more than one choice of 
the steady-state component. In applications one would prefer a steady state 
component having a relatively simple mathematical form. We show that in some 
cases a judicious choice of s^(0) makes this possible. The following theorem 
shows the properties of a class of steady-state components which are quite 
useful . 

Theorem . For any collection of 1 * w^ vectors s^(0), i = l,...,n, 

/V 

let the vector functions s^(t) satisfy 

s^(t) = s i (t-l)Q i + f i (t) + 1 = 

l 1 , . . . ,n , 

i.e. the vector functions s^(t) satisfy the basic stock equation (12). Then 
(a) the actual stocks at time t are 

i 

s i(t) = s.(t) + l (s k (0)-s k (0))M ki (t) , 
k=l 



23 



(b) 



oo 



l (s (t)-s.(t)) = 
t=0 



I 



k=l 



(s k (0)-s k (0))B ki N i; 



a 1 x vector having finite components, 

(c) s^(t) is a steady-state component of the stock vector s^(t), i.e. 

lim (s . (t)-s . (t) ) = 0. 

1 l 

t-xx> 

Before proving the theorem we explain why one might be interested in 
such a theorem. Part (c) of the theorem says that s^(t) is a steady-state 
component of the stock vector s^(t), and part (a) shows how the stock 
vector s^(t) can be expressed as the sum of a steady-state component and a 
transient component. Part (b) of the theorem says that the total over all 
periods of the difference between the stock vector and its steady-state com- 
ponent is a readily calculated finite vector. 

Such information can be useful when long-range planning has been done 
using an Equilibrium model. 11 As an example consider an organization which 
intends to change from its present size of 250,000 to a size of 200,000. The 
manpower planner may use an equilibrium model to develop policies that are in 
some sense optimal, and these policies will maintain the size of the organiza- 
tion at 200,000 people once the organization has been reduced to 
this size. So the equilibrium model tells the planner what to do once 
the size of the organization reaches the desired equilibrium level but it 
doesn’t tell him how to change the organization from its present 
level (250,000) to the desired equilibrium level (200,000). This problem of 
finding an optimal transition policy to go from present stock levels to a future 
equilibrium stock distribution is a very difficult one (see Chapter 4 , Bartho- 
lomew (1973)). One method for making the transition is to immediately implement 
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the hiring, promotion and attrition policies that have been derived from the 
equilibrium model. Because of the transient nature of the system these policies 
will eventually bring the stocks in the system to their equilibrium levels. 

/V 

In the theorem the vector functions s_^(t) play the role of what the 

stocks would be at time t if the system were in equilibrium. The stock 

vectors s.(t) indicate what the stocks will be at time t if we start with 
1 

the present stocks s_^(0) and implement the policies of the equilibrium model 
(which are reflected in the external flows, f^(t), and the transition matrices 
Qi> and A^). From part (a) of the theorem we may readily calculate the 

difference between actual stocks and equilibrium stocks in any grade and any 
period. If there is a penalty associated with having more people than the 
equilibrium stocks in the system, then part (b) of the theorem may be used to 
calculate the total penalty. Part (c) of the theorem assures the planner that 
the difference between the actual and equilibrium stocks does converge to a 
zero vector as the time parameter t increases. 

The proof of the theorem follows. 

Proof. By hypothesis the vector functions s^(t) satisfy the basic stock 
equation (12), so they must also satisfy the cumulative stock equation (13): 



t-1 t-1 

8 (t) = s (0)Q* + l f.(t-r)Q*; + l s i _ 1 ( t-r-l )P i_i Q i . 

r=0 r=0 



Of course the stock vectors s^(t) also satisfy the cumulative stock equation 
(13) , so we have, 



s.(t) - 7 (t) = (s 1 (0)-8 1 (0))qJ + l (s (t-r-l)-7 i _ 1 (t-r-l))P i _ 1 Qj. 

r=0 



When i = 1 this implies, 
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S 1 ( t ) = s^t) + (s 1 (0)-s 1 (0))Q 1 



_L 

= s 1 (t) + l (s k (0)-s* k (0))M k 1 (t), 



so we have shown that part (a) of the theorem is true when 1=1. Suppose 
part (a) of the theorem is true for grade i-1, i.e. 

s i-i (t) ■ 7 i-i (t> + j, ( \ <o) - s k <o))M k,i-i <,:) - 

k=l 



Then, 



i-1 



5 i _ 1 (t-r-l)-s i _ 1 (t-r-l) = l (s k (0)-s k (0))M k i _ 1 (t-r-l) , 



and 



t-1 i-1 



s.(t) - s.(t) = (s.(0)-s.(0))Q^ + l l (s k (0)-7 k (0))M k i _ 1 (t-r-l)P._ 1 Q^ 

r=0 k=l > 

■ <s i (0)-7 i (0))q‘ + T (s k ( 0 )-~ k ( 0 )) 1' \, i . 1 (t-r-l)P 1 .X 

k=l r=0 



From equation (10) in Section IV 

t-1 

l 

r=0 

so we have shown by induction that, 






i-1 



s i (t) - s.(t) = (s i (0)-s i (0))Q^ + l (s k (0)-s k (0))H ki (t) . 

k=l 

This proves part (a) of the theorem. 

From part (a), 



oo i 



I (s.(t)-s (t)) = l l (s (0)-s (0))K (t) 

t=0 1 1 t=0 k=l k R kx 

= l (s (0)-s (0)) l (t) 
k=l fc t=0 K1 
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" l (s k (0) -\ (0))B kl N i' 

k=l 

a 1 x w_^ vector having finite components. 
The last step above follows from equations (6) and (8) of Section IV. This 
proves part (b) of the theorem. 

Part (c) follows from the fact that the sum in part (b) is finite, and 
the proof of the theorem is complete. 

The utility of this approach depends on our ability to find vectors 
s_^(0) such that the vector functions s^(t) are simple and readily calculated 
Some examples follow. 

1 . Fixed External Flows . 

The equilibrium models previously mentioned enjoy some popularity in 
military manpower planning in the United States (see for example, RAND Corpora- 
tion (1973)). The rationale underlying the use of such models is that one 
should determine the organization structure and the policies to maintain this 
structure which are optimal (or M least infeasible"). Among the policies 
derived from an equilibrium model is the hiring policy. This has the form. 



f i(t) = f i, t = 1,2, . . . , 

i = 1, . . . ,n 



where the vector of the number of people to be hired into the states in grade 
each period, f^, is specified from the equilibrium model. 

Define, 



Sl (0) = fjN^ 



Then using (12) it is easy to show that 

s^(t) = ^x N l f ° r 
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Thus, from the theorem 



s l< t ) = S^t) + (s 1 (0)-s 1 (0))M 11 (t) 



= + (s 1 (0)-f 1 N 1 )Q 1 . 



Now recursively define. 



s 1 = s 1 (t) = 



s. = (f.+s. -.P. ,)N., i = 2,...,n. 
1 1 l-l l-l l 



(14) 



It is straightforward to verify that these s^ satisfy the basic stock 
equation (12), so we have from the theorem, when f^(t) = f^. 



s.( t ) - 7. + I (s k (0)-J k )M kl (t) 

k=l 



The steady-state component can also be written, 

■ j, f k B ki N i> i-l, •••."• (15) 

k=l 
i 

Note that £ ^k^ki a non ” ne S at ^ ve 1 x w^ vector, so the limiting 
k=l 

vector of stocks in grade i must be a non-negative combination of the rows 
of N^. Thus, in general, not all non-negative 1 * w^ vectors are possible 
limiting stock vectors under constant external flows. 

2 . Linear Growth of External Flows . 

In this section we consider the case in which the number of people hired 

into each state increases by the same amount each period. Such a hiring policy 

may not be natural over a long period of time, but it may provide a simple 

approximation to planned hiring policies. 

Let the 1 x w. vector f. be the increase in the number hired into 
l i 

states with FC i each period. Then the external flow vector for FC i is. 
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Let, 



f i (t) = tf ± , t = 1,2, . . . , 

1 !)••• )!!• 



Sl (0) = -f 1 N 1 Q 1 N 1 . 



Let the vector function s^(t) satisfy the basic stock equation (1) . 



s 1 (t) = s 1 (t-l)Q 1 4- f 1 (t). 



Using the identity N^Q^+I = one can show that 



s i (t) = tf i N i - Wi ' 



Thus from the theorem. 



s x (t) = tf 1 N 1 - f 1 N 1 Q 1 N 1 + (s 1 (0)+f 1 N 1 Q 1 N 1 )Q 1 . 



We note that s^(t) is of the form 



s^t) = tL^ + C x 



where L, = f,N n is a 1 * w. vector, 
111 i 



and CL = -f .. N.. Q.. N n is a 1 * w. vector. 
1 1 11 1 l 



Consider some FC it {2,...,n}. Suppose that 

Vi (t) = tL i-i + C i-1» 

where L. - and C, - are 1 x w. vectors, 
l-l i-1 i-l 

Using the identity 

(tf .N.-f.N.Q.N.)Q. + (t+l)f. = ( ( t+1 ) f N -f N Q . N ) , 
111 1^1 11 1 111111 

one may show that if 



s.(t) = tf.N. - f.N.Q.N. +s. (t-l)P . -N. - L P N Q N , 
i li i ii i l-l i-l i i-l i-l ill 
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then s^(t) satisfies the basic stock equation (12). Note that s^(t) has 
the form, 

s . (t) = tL. + C . , 
i 1 l 

where , 



L. 

l 



f.N. + L. -P. -N. 

l i i-l i-l i 



= (f.+L. JN., 

l i-l i-l l 



(16) 



and , 



C. 

i 



-(f.+L. ,P. )N. Q . N . 

l i-l i-l li i 



a i-r c i-i )p i-i N i 



- -((L. ,-C. ,)P. , + f.N.Q.)N.. 
l-l l-l l-l 1 1^1 1 

Thus we have shown that when the external flows grow linearly the steady-state 
component of the stocks also grows linearly. 

By recursive substitution in (16) we have, 



L i ■ J, £ AA- 

k=l 



Note that this vector gives the expected number of visits to states with FC 
i of f^= f^(t+l) - f^(t) entrants with FC k, k = l,...,i. That is, the 
growth in the stocks with FC i each period, L^, equals the expected number 
of visits to FC i of the growth in the external flows each period in the 
FC ! s less than or equal to i. 

Both L. and C. have the fundamental matrix N. as a right factor, 

li l 

so the steady state component of the stock vector, s^(t) must be a non- 
negative combination of the rows of N^. This same result was observed in 
the case of constant external flows. 

In summary we have shown that by choosing 



s . (t) = tL . + C . 
l i l 
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where 



L. = f,N- when i = 1, 
ill 



= ( f i+ L i-l P i-l )N i> 1 = 



and 

C. = when i = 1 , 

i 1 11 1 

= - ( (L. -C. _)P. - +f . N . Q . ) N . , i = 2 , 

i-l i-l i-l i i x i i 

then from the theorem the stock equation may be written 



s i(t) = s^t) + l (s k (0)-s k (0))M ki (t) . 

k=l 

3. Geometric Growth of External Flows . 

In this subsection we show that geometric growth of external flows leads 
(eventually) to geometric growth of the stocks. We consider the case in which 
the external flows into the states in grade i are proportional to a known 
vector f^ and grow geometrically at a rate 6^. Thus, 



f.(t) = 0.f., 

i li 



t = 1,2,..., 
i — 1 , . . . ,n 



0 . > 0 . 

l 



When 0 < 0_^ <1, the external flows contract rather than grow. 

If 0^ is not an eigenvalue of for k ^ i ^ n we may define, 

W ° <* -f Q i ) ‘ 1 - 

k 

If the states in grade i have the 0-1 visiting property then all eigenvalues 
of Q_^ are zero and thus 0^ > 0 is never equal to an eigenvalue of in 

this case. 

The following identity will be useful : 
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vv - i <f v r 



r=0 



W\ = e k i ( f V 

r=0 k 



r+1 



* 9 k ( - I+ W> 



Define, 

s^O) = f 1 N 1 (0 1 ). 



Then it can be shown that if 



s i(t> = e i f 1 N 1 ( 0 1 ) » 

<" w 

then s^(t), t = 0,1,..., satisfies the basic stock equation, and from the 
theorem, 

s l (t) = G l f l N l (e i ) + ( s 1 (°)- f 1 N 1 (0 1 )) M 11 (t)* 

Note that the steady-state component of the FC 1 stock vector grows geometri- 
cally at the same rate as the external flows into FC 1. 

Define, 

i-1 

B (0 ) = n (N (0 )P ), ISkSiSn. 
ki k , m k m 

m=k 

Then it can be shown that if 



(o = I 

k=l 



6 k" (i " k)f k B ki ( 0 k )N i (e k ) 



then s^(t), t = 0,1,..., satisfies the basic stock equation (12). Note that 
in the limit the stocks with FC i grow geometrically at the rate of the 
largest 0 where k ^ i. 

K. 
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Define , 



0 M = max{0 k ; k=l , . . . ,i} , 

The steady-state component of the stock vector is not in general a non-negative 

combination of the rows of N. (as was the case with constant external flows 

1 

and linear growth of external flows). Rather the steady-state stock distribu- 
tion is a non-negative combination of the rows of The rows °f (0^) 

need not be non-negative combinations of the rows of N^, so the limiting 
stock distributions that are possible under geometric growth of external flows 
need not be the same as the limiting stock distributions under constant external 
flows and linear growth of external flows. 
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